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PREFACE 



It is technically possible to compare student, departmental, or campuswide 
data^among institutions. It is not necessarily appropriate or meaningful 
to do so. The imagination is capable of creating infinite details and 
myriad examples of the implications of the issue of data comparability. 
The authors have chosen, instead, to adcress themselves first to the philo- 
sophical issues of comparability, which they deem basic to such necessary, 
but subsequent discussion of details and illustrations. 

In this paper, then, the authors present a philosophical statement on the 
problem of data comparability. The discussion is intended to provide guid- 
ance in determining the direction of the Information Exchange Procedures 
project of the National Center for Higher Education Management Systems at 
WICHE. 



The reader's response is solicited. 



INTRODUCTION 



The major resources made available to higher education during the expansive 
decade of the 1960s enabled relatively painless decisions on resource 
allocation to be guided more by concern for desired enrollment and academic 
program development than by concern for the efficient use of resources. Now 
the operative forces of the 1970s demand a reordering of higher education's 
administrative priorities. The public's open-ended financial commitments 
to welfare, to public medical care, to unemployment, to the military, and to 
space programs are requiring an ever-increasing share of the state and 
national treasuries. Virtually every area of public expenditure, including 
higher education, is being cut back, is being maintained at current levels, 
or is experiencing a slower rate of increase in the face of rising demands 
for service. 

Mounting frustrations with the rapidly increasing costs of higher education 
lead* the appropriators and allocators of scarce public funds to look for more 
information that would help them with their difficult decisions. They want 
to know how an infusion of funds will benefit people and programs or how a 
withdrawal of funding will adversely affect those people and programs. They 
must keep in mind both individual and societal needs in their own states, 
and they must necessarily consider how the productive utilization of funds 
administered at one institution compares with fund utilization at another 
institution. 



/ 
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Legislators, funding agencies, chief executive officers, governing boards, 
and donors not only are experiencing what Cheit calls the "new depression 
in higher education." but also must bear the responsibility of guiding 
and, managing higher education in these times. The question is not whether 
higher education will continue to exist and to receive public support, but 
rather, how much support is necessary, for what activities, for whom, at 
what price, and who will make the decisions? Many of these decisions are 
value judgments whose answers are found in one's mind and heart and not in 
.the data base of an institution. Nevertheless, the legitimate concerns of 
legislators, funding agencies, chief executive officers, governing boards, 
and donors include necessary questions about how effectively institutions 
are managing their resources and what costs are reasonable for certain kinds 
of activities. 

It is primarily for people with these general concerns for proper use of 
resources that this paper is written. 

BACKGROUND 



The basic objective of the National Center for Higher Education Management 
Systems (WICHE-PMS) is to improve the management capability of institutions 
and state agencies of higher education. Virtually all the activities of 
NCHEMS (WICHE-PMS) began in response to management needs of higher education 
While certainly not comprehensive, the following management concerns and 
needs require particular attention: 
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Better understanding of the goals, aspirations, and needs of the 
constituents of higher education to determine objectives and to 
outline better methods of resource allocation. Further, there 
is urgent need to understand the market for college graduates in 
order to assure continued financial support. 

Better identification of institutional objectives. Failure to do 
do is bad management in a public institution; success in doing so 
will significantly improve higher education's credibility. In 
fact, r clear expression of objectives may be higher education's 
highest priority need, 

a. A necessary element of this identification must be an increased 
understanding of how these objectives meet the goals and 
aspirations of the institution's constituents and an increased 
specificity of the objectives to facilitate better planning. 

b. Higher education must express its objectives in terms that 
people outside the institutions of higher education can under- 
stand. 

c. Finally, such identification must lead to better understanding 
of which information associated with the decision-making 
processes appears to be the most credible to the constituents 
of higher education in order to be able to express precisely, 
believably, and in language that is understood the objectives 
and costs associated with higher education. Administrators 
must be able to communicate in the same language to legislators 
and funding agencies. 



Clearer delineation of alternate courses of action to meet 
objectives and a comnon understanding of alternate descriptions 
of objectives . 

Workable criteria to tell administrators, and others, whether 
objectives have been achieved and, if not, why not. 

Clearer understanding of the costs associated with achieving 
specified objectives using alternative courses of action. 

Better methods of evaluating the costs of programs in relation to 
their accomplishment of institutional or public objectives. This 
includes data on comparative mixes of the resources used by 
programs and the various characteristics of the outputs produced 
by the same programs. 

Better understanding of the interrelationships between restricted 
and unrestricted funds and a confidence that funds from external 
sources are being maximized without (a) violating the objectives 
of the institution or (b) conmitting other institutional resources 
by accepting "free money" and thus subverting the normal resource 
allocation process. 



( 8. Better understanding of the resources tivailable for higher educa- 

tion and their impact. For example, which choice of activities 
will attract the necessary resources to accomplish alternate 
sets of desired objectives? Which resource acquisitions will 
apply constraints upon existing or potential resources? Which 
sets of resources will attract other resources? How are these 
affected by the magnitude of the resources received? 

9. Better methods of amortizing all costs so as to associate more 
equitably the cost burdens with benefits received over time. 

No absolute standards apply to the measurement of objectives, to efficient 
resource use, to optimal resource acquisition, or to the other concerns 
raised by educational administrators. In the absence of absolute norms, 
planners turn to comparative measures of costs, goal achievement, and 
different mixes of resource use as one possible method of evaluating the 
effectiveness and efficiency of a given activity or program. For example, 
faculty salaries, student/faculty ratios, and tenure ratios are computed 
at similar institutions to determine whether the number of faculty members 
at Institution A should be increased and whether their salaries are 
competitive. Such comparisons have long been made and will continue to 
be made by institutions themselves and by their funding agencies. 

One of the purposes of NCHEMS (WICHE-PMS) is to make meaningful comparisons 
possible, recognizing that not all possible comparisons are meaningful. 
The question of whether or not comparisons are possible or technically 
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feasible is a question of compatibility ; some guidelines for compatibility 
are given in the following section. The question of whether or not compar- 
isons are meaningful is a question of comparability ; some guidelines for 
comparability are also given in a subsequent section. While the require- 
ments for compatibility are largely technical, the requirements for 
comparability are far more judgmental, because comparability, like beauty, 
is in the eye of the beholder. 

GUIDELINES FOR COMPATIBILITY ..^ 

In general, we say that one set of data is compatible with another set of 
data if the elements of one set are capable of orderly, efficient integration 
and operation with elements of other sets of data. Data derived from one 
institution (or activity) are compatible with data derived from another 
institution (or activity) to the extent that both data sets are composed of 
data elements that are (1) defined in exactly the same way, (2) measured in 
exactly the same way, and (3) aggregated and summarized by the same procedures. 

For example, if space utilization of two institutions is to be compared, 
the kinds of spaces and the use of space must be measured compatibly in the 
two institutions. This means that both institutions should use the same 
definitions of space types (classroom, class lab, office), room capacities, 
use categories (lecture, seminar, research, office, maintenance), time periods 
of use, and all the other basic characteristics of physical facilities. In 
addition, both institutions must use the same measures of the space, including 



differentiation between assignable square feet for instructional use or some 
specified purpose and outside gross square feet. Finally, both institutions 
must aggregate and report their space data in exactly the same way. The 
analysis of physical facilities is one in which "standard" definitions, 
measurements, and aggregation conventions have existed for some time and 
have produced compatible measures among institutions. Hov/ever, most other 
areas of institutional activi cy do not have a tradition of compatible data. 
One of the goals of the NCHEMS (WICHE-PMS) Data Elements Dictionary and 
Program Classification Structure is to establish, thrcugh consensus, a set 
of data element definitions and aggregation procedures that will lead to 
"NCHEMS (WICHE-PMS) compatible" data systems for major areas of institutional 
activity. In general, information systems are NCHEMS (WICHE-PMS) compatible 
to the extent that they use NCHEMS (WICHE-PMS) standard data elements and 
NCHEMS (WICHE-PMS) standard aggregation and analysis procedures. 

Having compatible measures, however, meets only one (and perhaps the simplest) 
condition necessary for comparability. 

GUIDELINES FOR COMPARABILITY 

While it is technically possible to compare student, departmental, or campus- 
wide data produced by NCHEMS (WICHE-PMS) compatible information systems for 
two institutions, it is not necessarily appropriate or meaningful to do so. 



The comparison of even compatible data should be based on either consensus 
or an executive decision on: 

K the purposes of the comparison 

Z. the description of the institutional activities 

or outputs being compared 
3* the relations between resources consumed and 

activities performed 
4. the probable usefulness of the information obtained 
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In describing the purposes of the comparison, the funding agency or administa- 
tor making the comparison should articulate his objectives and show how these 
objectives relate to the comparison. At this point, the managers of the 
progra:ns that are being compared might argue that the purposes of their 
activities are different from the objectives indicated by the funding agency. 
This conflict must be resolved either by agreement or by executive decision 
before a meaningful comparison can be conducted. 



For example, if a state's legislature is primarily interested in raising the 
general level of language ability of the state's population, then legislators 
might want to compare the average increase in verbal ability scores achieved 
by students in coranunity colleges, state colleges, and research-oriented 
universities. On the other hand, academic deans might argue that some 
English departments serve special purposes, including, for instance, the 
training of Ph.D.'s for other English faculties, and, therefore, a comparison 
of the costs associated with the increases in verbal ability scores should 
not be made among conmunity colleges, state colleges, and major universities. 
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This is a fairly typical example, and the conflict of the objectives of the 
comparison with the respective objectives of the programs being compared must 
be resolved before all parties to the comparison will respect its results. 
The second point is that the characteristics (parameters, proxies) in terms 
of which two activities or institutions are being compared must be the same. 
If one decided that students completing an engineering program could be described 
by their Graduate Record Examination scores, graduate admission percentage, 
and average starting salaries, then one should collect the same compatible 
data from any other institutions that are candidates for comparison. This 
means that the activity characteristics used for comparison should be as 
comprehensive as possible within the purposes of the^xpmparison. Furthermore, 
the activity characteristics should include the timing and scheduling of 
activities of the institution. For example,' one should avoid the pitfalls of 
comparing nine-month academic appointments with annual appointments in private 
industry or one-time-only extension courses with major or academic course 
sequences extending over several years. 

Finally, all parties to the comparison should understand the relationships 
between the resources consumed and the activities performed. Many edu- 
cational activities are joint processes that consume a wide variety of 
resource inputs (dollars, space, personnel) and provide a variety of out- 
puts (educated young people, new inventions or research findings, service 
to other programs, community services) in such a manner that they are 
inextricably linked. Attributing costs in joint process situations in 
any meaningful way is very difficult; arbitrary cost proration procedures 
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may be very misleading. Therefore, comparisons between two joint process 
activities or between a joint process activity and a single purpose activity 
should be done very carefully with full cognizance of the particular 
relationships within the two sets of activities. 

Another important consideration in determining the comparability of two 
activities or institutions is whether or not they consume comparable 
resources . It is very tempting to assert that "a dollar is a dollar is 
a dollar" because all dollars are interchann?able. But their purchasing 
powers in different markets are certainly not. For example, the salaries 
paid to faculty and administrators of religious institutions are typically 
below salaries paid by public institutions because sectarian schools often 
recruit their faculty in a very limited market of individuals especially 
desirous of serving in that kind of institution. Other examples are the 
regional variations in construction costs and clerical wages. In general, 
for meaningful comparisons the institutions being compared should purchase 
their inputs in the same market, or an adjustment should be made to account 
for the price differentials in the respective markets. This means that 
most of the unit prices of commodities purchased by educational institutions 
will be approximately the same, with perhaps some variation in personnel 
costs reflecting the differing nonmonetary rewards associated with emplo.yment 
in different situations. 

A further consideration in the relationships between resources consumed and 
activities performed are the oEgrating constraints that an institution faces. 
Restricted fund sources often influence the decision to undertake one activity 
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instead of another. Accounting conventions, internal recharges, traditional 
income-sharing procedures (e.g. federal overhead recovery divided evenly 
between the governing board and the state government or general endowment 
apportioned among the various schools on the basis of the book values of their 
individual endownents), and many other factors all influence the mix of 
activities chosen by an institution. These factors should be explicitly 
stated and understood by all parties to fhe comparison. 

These comments are not meant to inhibit meaningful comparisons but to facili- 
tate them by pointing out some of the major guidelines that one should follow 
in constructing an appropriate comparison. These guidelines are summarized 
below. 

1. The parties to the comparison should be aware of and agree to the 
purposes of the comparison to insure credibility of the results. 

2. The comparison should be based on similar comprehensive descrip- 
tions of the activities for the purposes of the comparison. 

3. The parties to the comparison should understand the relationships 
between the costs and the levels of activities with particular 
attention given to: 

a. joint products (many outputs from one activity) 

b. characteristics of the inputs (are they comparable?) 

c. operating constraints (restricted fund sources) 



Adhering to these guidelines will not insure comparability, but such adherence 
will avert many inappropriate and potentially misleading comparisons. In 
other words, good judgement is needed in addition to these three major guide- 
lines to establish meaningful comparisons whose full implications can be 
realized in practice. 

STANDARDIZATION vs. REGIMENTATION 

A fear often expressed by faculty and academic administrators is that stand- 
ardization of educational data will inevitably lead to regimentation of 
educational offerings. Some of this fear would be assuaged (or possibly con- 
firmed) by the first guideline, which would make the purposes of the comparison 
explicit. At least the uncertainty of motives would be reduced. But a signi- 
ficant residual fear would undoubtedly remain, a fear stemming from a mis- 
interpretation of the role of standards. The existence of a standard for 
length (the meter) does not result in everything's having the same length. 
Because a carpenter uses a measuring rule to mark his lumber does not mean 
that every cabinet will be the same size but that the top and bottom of a 
given cabinet will match. Standards of volume and weight insure that a shopper 
knows how much she is buying in a supermarket, but they do not force all foods 
to be alike. In other words, standard measurements can enable diversity and 
provide the tools to avoid homogenization . 



CONCLUSION 



Whether or not everyone agrees on the notion of comparability, comparisons 
will be made. Difficult choices will be made. The complex nature and 
diversity of higher education, coupled with the ever-present, ever-Increasing 
shortage of both public and private funds, make choices in spending and allo- 
cation inevitable. The range of disagreement over resource allocation can be 
narrowed with the wise use of comparable data by identifying and explaining 
factual differences between programs. Such use would enable administrators, 
legislators, governing boards, and others to focus much more of their 
attention on the judgmental variables in their decisions. It is necessary, 
therefore, that we reach common definitions and understanding on all sides 
of the issue of resource use. The future adequacy of funding of higher 
education may depend on the outcoine of this struggle for understanding. 
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